Replaces existing components/execution/ with Langfuse-aligned observation
viewer. Independent module with 5 React Contexts, iterative tree building,
virtualized tree/timeline views, and WebSocket real-time streaming.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

…alignment

Complete observation trace viewer module (components/observation/) with:
- Iterative bottom-up topological tree building (10k+ node safe)
- Virtualized tree + Gantt timeline views via @tanstack/react-virtual
- 4 React Contexts (Data, Selection, ViewPrefs, JsonExpansion)
- WebSocket live stream + historical replay modes
- snake_case → camelCase normalization layer for backend alignment
- Backend traces API: add completion_start_time, tool_call_names, metadata
- Replace old ExecutionPanel with DebugPanel in CodeEditorPage and TestLab
- 30 tests across tree-building, tree-flattening, timeline-calculations

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

api-client already prepends /api/v1, so passing /v1/traces resulted in
/api/v1/v1/traces (404).

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

LangChain sometimes passes serialized=None to on_chain_start,
on_llm_start, and on_tool_start. Guard with (serialized or {}).get()
and fall back to kwargs["name"].

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Full Langfuse-parity rewrite with OTel-native architecture,
LiveSpanProcessor for streaming, and unified SpanProcessor pipeline.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Fix: fields already exist (#1), async bridging pattern for processors (#2/#3),
_live_processors registry (#4), CHAIN/AGENT classification rule (#5),
on_agent_action type-mutation strategy (#6), trace row lifecycle (#7/#8),
OTel context API note (#9), dependency declaration (#10),
state field documentation (#11), frontend protocol coordination (#12),
example envelope consistency (#13), add_event flat attrs (#14),
_reset vs _detach_span ordering (#15).

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Fix: trace row lifecycle ownership (C1/C2), method name consistency (C3/C4),
stale on_end sketch (C5), event_loop sourcing (I1), extractor API migration
table (I2), sync call shape (I3), trace_complete before shutdown (I4),
async context propagation note (I5), frontend change list (I6),
thread-safe seq counter (I7), drain_future type (S1),
parent observation_id recovery (S2), DB visibility note (S3),
_track_run defensive fallback (S4).

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Replaces the ambiguous 3-state model (ready/failed/retired) with an
explicit 5-state machine: ready → active → superseded, plus failed and
retired. Release status now reflects "who is live" and "who can roll back"
directly, instead of relying on Agent.active_release_id alone.

- Backend: extend agent_release_status enum + Alembic backfill;
  rewrite activate/retire and add unpublish_release; new POST /unpublish
  endpoint; tighten status literals to Literal[...]
- Frontend: shared ReleaseStatusBadge + canRollback/canRetire helpers;
  rewrite release stage with show-archived toggle and Unpublish action;
  fix isActive bug in useDeploymentHistory (was checking 'ready');
  i18n keys for new statuses and unpublish flow

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

14-task TDD plan covering:
- OTel package skeleton (LiveSpanProcessor, ObservationSpan, PersistenceProcessor,
  BroadcastProcessor, ObservationTracerProvider)
- langchain_utils (message serialization, usage normalization, model extraction,
  chain classification)
- ObservationCallbackHandler full rewrite with all 18 LangChain hooks
- ObservationCollector OTel-backed rewrite
- Old file deletion (writer.py, broadcaster.py, SpanHandle)
- Extractor adaptation (cli, copilot, file_tracker)
- Engine integration updates (orchestrator, graph_engine, code_engine, copilot_engine)
- Verification + frontend coordination stub

Reviewed by plan-document-reviewer; 3 critical issues fixed:
- Removed non-existent execution_runner.py from inventory
- Made PersistenceProcessor tests async (running loop required for drain task)
- BroadcastProcessor maintains _otel_span_id_to_observation_id mapping for
  parent resolution in span_open/span_close/llm_token envelopes

Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…SpanProcessor

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

…an writer

Bridges OTel's sync SpanProcessor callbacks to async SQLAlchemy via an
asyncio.Queue + drain loop. Converts finished spans into Observation rows
with batched flushes, tracks parent observation_id via OTel span_id mapping,
accumulates token/cost/error aggregates, and persists non-stream events as
child observations.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

- PersistenceProcessor._flush: add session.rollback() and session.close()
  to prevent DB session leaks on flush failure
- BroadcastProcessor.on_event: use getattr for parent resolution to fix
  pyright errors accessing .parent on Span interface type
- Remove unused imports from broadcast_processor test

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

… lifecycle

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

… normalization, model extraction

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

…gChain hooks

Replace the old ObservationCollector/SpanHandle-based handler with an OTel-native
implementation that covers all 18 async callback hooks: chain (start/end/error),
LLM (start/end/error/new_token), chat_model (start), tool (start/end/error),
retriever (start/end/error), agent (action/finish), retry, and text.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

- Import Tracer from opentelemetry.trace (API) instead of opentelemetry.sdk.trace
- Make serialized optional (dict | None) in on_chat_model_start
- Remove unused test imports

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

…ced by OTel pipeline

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

… execution_runner

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Publish froze the current draft but never created a new one, leaving
agent.current_draft_version_id pointing at a frozen version. Canvas
auto-save then hit 400 AGENT_VERSION_FROZEN repeatedly.

- Add AgentVersionService.fork_draft_from() with optional overrides
- Publish flow now forks a fresh draft after freeze
- update_version transparently forks when it hits a frozen version
  (handles existing bad state and publish/save race)
- Frontend save path detects version_id change and switches graphStore
  to the new draft, preventing repeated forks

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

- Import Tracer from opentelemetry.trace (API) instead of SDK in provider.py
- Split PersistenceProcessor.shutdown into sync (OTel GC-safe) and async_shutdown
- Guard sync shutdown against closed event loop at process exit

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

useSessionStream kept events/connected/error as bare state that was
never cleared on a sessionId change, so the previous session's events
bled into the next one and after_seq carried over. State is now tagged
with its owning sessionId and gated on read, lastSeq resets per run, and
a runId guard drops stale async writes. The session page resets its
view state (loaded events, drawers, input) on session change and guards
loadEvents against out-of-order responses.

Adds a regression test that drives the hook with fake timers and asserts
synchronously (testing-library's waitFor cannot be used under fake
timers) proving events and after_seq reset when switching sessions.

Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>

…a-table updates

- deploy: expose JOYSAFETER_ENVOY_XDS_MODE (default filesystem) so the Envoy
  LDS transport (filesystem vs Delta gRPC xDS) is configurable per deployment
- frontend: managed skills page + shared data-table tweaks, i18n strings, deps

Note: the Rust orchestrator xDS implementation lives under
backend/app/joysafeter_orchestrator_rs/ which is gitignored (local experiment,
do-not-publish), so its source is intentionally not part of this commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…oxy vars

- deploy: Envoy entrypoint now also seeds an empty cds.json alongside lds.json,
  so a fresh container has a valid CDS file when the bootstrap references it
- runner: when the socat egress bridge starts, export http_proxy/https_proxy
  and ALL_PROXY/all_proxy (lowercase + ALL_* variants) in addition to the
  uppercase HTTP_PROXY/HTTPS_PROXY, so tools that only honor the lowercase or
  ALL_PROXY conventions also route through the Envoy egress boundary

Verified locally against a limited-networking sandbox: LLM base URL is
rewritten to the Envoy egress placeholder, no real credential enters the
sandbox (the key present is a non-secret placeholder; the real key is injected
by Envoy at egress), and non-allowlisted hosts are rejected with 403.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ng sandbox deadloop

Security:
- Runner token scrub: entrypoint wrapper (runner-entrypoint.sh) saves token to
  temp file, unsets env var, then execs runner. Runner reads from file and
  deletes it. docker exec env no longer exposes JOYSAFETER_RUNNER_TOKEN.
- Codex entrypoint updated with same token scrub logic.
- claudecode/native Dockerfiles switched from direct ENTRYPOINT to wrapper.
- Runner also does defense-in-depth remove_var + remove_file after first connect.

Envoy:
- write_file_in_envoy: touch parent dir after mv to fix inotify on Docker bind
  mounts (remote server Envoy was not detecting lds.json changes).
- HTTP listener: stream_idle_timeout=0s (disable 5m default) + route timeout=0s
  (disable 15s default) — fixes long LLM streaming responses being truncated
  mid-stream through Envoy proxy.

Orchestrator:
- Fix creating sandbox deadloop: S10 branch now destroys stale creating sandbox
  before re-provisioning, preventing idx_csb_active_session_unique constraint
  violation retry loop.

Deploy:
- TZ=Asia/Shanghai added to all services in both compose files.
- frontend bun.lock updated for @tanstack/react-table dependency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ured error

The update_environment failure path read env.id after db.rollback() had
expired the ORM instance, which triggers a synchronous lazy-load under async
SQLAlchemy (MissingGreenlet) — so clients received a greenlet error instead of
ENVIRONMENT_IMAGE_BUILD_FAILED. Use the env_id parameter instead.

Also repair the lifecycle tests: retarget the stale _validate_and_build_image
monkeypatch to _build_image_update (renamed), and capture task.id / agent.name
before the call so the conflict-path rollback's object expiration doesn't break
the post-call assertions.

Co-Authored-By: Claude-Opus-4.8[1m] <noreply@anthropic.com>

The Rust orchestrator — previously gitignored as a local experiment — is
now tracked in version control. It provides:

- gRPC AgentBridge server for sandbox-runner connections
- Task scheduling with per-session sandbox resolution (reuse/pool/create)
- Sandbox lifecycle management (idle sweep, provisioning poll, orphan cleanup)
- Envoy network isolation with pluggable LDS/CDS backends:
  - Filesystem xDS (lds.json watched by Envoy inotify)
  - Delta gRPC xDS (incremental ADS, O(1) per sandbox update)
- Per-sandbox credential injection (LLM keys, MCP tokens, git tokens)
  via Envoy header mutation — secrets never enter the sandbox container
- Provider support: Anthropic, OpenAI, Google Gemini, Azure OpenAI
- DB-based LDS recovery on orchestrator restart
- Container hardening: cap-drop ALL, no-new-privileges, pids-limit
- Event batching and Redis stream publishing
- HA coordination via Redis (instance heartbeat, sandbox ownership)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…rator-rs)

The Python orchestrator (backend/app/joysafeter_orchestrator/) has been fully
replaced by the Rust implementation (backend/app/joysafeter_orchestrator_rs/).
API and Worker services that previously imported from this package now get
None from the get_* helpers and fall back to Redis-based communication with
the Rust orchestrator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Memory store real-time updates now propagate from the API to running
sandboxes correctly:

- API broadcasts memory_update to the orchestrator instance that OWNS
  each sandbox (resolved via joysafeter:sandbox_owner), not the API's
  own instance_id — the two are different processes and the old channel
  was never listened to.
- Broadcast SQL now targets sandboxes whose session status is not
  ended/error (was incorrectly limited to active/awaiting_input, so idle
  sandboxes never received updates).
- CommandListener handles the memory_update broadcast and fans it out to
  all peer sandboxes sharing the store via MemoryStoreSubscribers.
- Runner applies updates on both the FUSE and Docker bind-mount paths.

Removed the redundant memory preloading in sandbox_resolver that wrote a
stale duplicate to /workspace/mnt/memory. Memory now lives only at the
canonical /mnt/memory/{slug} mount (per the Managed Agents spec); the
runner writes initial files there from the SetupSandbox payload and
real-time updates target the same path.

Also completes the Python orchestrator removal: shared getters, proto,
RuntimeConfig, SessionBroadcaster and ImageBuilder moved to
joysafeter_shared/orchestrator_bridge; all API/Worker imports repointed;
joysafeter_orchestrator/ deleted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>